# Large-scale Corpus

A Chinese PEGASUS-large model specialized in text summarization tasks, fine-tuned on multiple Chinese summarization datasets

ERNIE 3.0 is a large-scale knowledge-enhanced pre-trained model for Chinese language understanding and generation, with the mini version being its lightweight implementation.

BERT-large variant pretrained on large-scale scientific paper collections with 340 million parameters, specializing in scientific literature comprehension

ProcBERT is a pre-trained language model specifically optimized for process texts, pre-trained on a large-scale corpus of process texts (including biomedical literature, chemical patents, and cooking recipes), demonstrating outstanding performance in downstream tasks.

IndoBERT is a state-of-the-art language model developed for Indonesian based on the BERT architecture, trained using Masked Language Modeling (MLM) and Next Sentence Prediction (NSP) objectives.

A massive German language model trained on the cleaned German Common Crawl corpus (GC4), totaling approximately 844GB, which may contain biases.

A Wav2Vec2 base model pretrained on the Dutch subset of the VoxPopuli corpus, suitable for Dutch speech recognition tasks.

AraBERT is a pre-trained language model based on Google's BERT architecture, specifically designed for Arabic natural language understanding tasks.

This model was pre-trained on a 250-million-word Chinese corpus using the MobileBERT architecture, with a training period of 15 days, completing 1 million iterations on a single A100 GPU.

XLM-RoBERTa is a multilingual model pretrained on 2.5TB of filtered CommonCrawl data across 100 languages, trained with a masked language modeling objective.